ZERO-SHOT COMPOSITIONAL EVENT DETECTION VIA GRAPH MODULAR NETWORK

Humans are known to have the capability of understanding events by composing different atomic concepts, even for event types that have never been seen before. However, event detection has been so far treated as a sequence tagging problem in literature. Despite the increasing accuracy obtained on benchmarks such as ACE, current supervised sequence tagging models lack the compositional generalization ability. We present a model that is able to achieve zero-shot compositional generalization for event detection. Our model, named compositional graph modular network (CGMN), proposes two separate graph neural networks to obtain compositional semantic representations for sentences and events respectively. Meanwhile, it ties graph-based event representations with the weight parameters of an event matching layer, so that the semantic representations for sentences and events can be connected with each other, thereby achieving zero-shot recognition of new events using only their constituent atomic concepts. Our experiments on the ACE 2005 dataset as well as our collected Twitter event dataset show that, CGMN significantly outperforms state-of-the-art event detection methods on unseen classes and demonstrate strong zero-shot compositional generalization capabilities.M.S

Risk, return and market condition: a new functional-beta capital asset pricing model

In this research, we will focus on investigating the relationship between risk and return. We will propose a new model which leads to a more sensible approach to modelling the relationship between risk and return under different market conditions. It is an extension of the traditional single-index capital asset pricing model (CAPM) which reads as: The return R[subscript]i on individual Security i can be decomposed into the specific return α[subscript]I + ε[subscript]i (expected specific return α[subscript]i and random specific return ε[subscript]i) and the systematic return β[subscript]iR[subscript]m owing to the common market return R[subscript]m.In our new model, we suggest a functional-beta single-index CAPM, extending the work of three-beta CAPM (Galagedera and Faff, 2004) that takes into account the condition of market volatility. Differently from the three-beta CAPM, we allow β[subscript]i changing functionally with the market volatility σ[subscript]m, which is more flexible and adaptable to the changing structure of financial systems. The main contributions of this thesis are summarised as follows:• A new functional-beta CAPM, taking into account the conditions of market volatility, is proposed under the framework of widely applicable data generating processes of near epoch dependence (NED).• A semi-parametric estimation procedure based on least squares local linear modelling technique under NED is suggested with the large sample distributions of the estimators established.• Simulation study is fully made, illustrating that the suggested estimation procedure for the proposed functional-beta CAPM under near epoch dependence can work well. It provides reasonable estimates of the functional beta in the condition of moderate market volatility.• By using a set of stocks data sets collected from Australian stock market in the past ten years, empirical evidences of the functional-beta CAPM in Australia are carefully examined under both nonparametric and parametric model structures. Differently from the three- or multi-beta (constant) CAPM in Galagedera and Faff (2005), our new findings show that the functional beta can be reasonably parameterized as threshold (regime-switching) linear functions of market volatility with two or three regimes of market condition. In the condition of extreme market volatility, a threshold functional-beta CAPM is suggested.The CAPM provides a usable measure of risk that helps investors determine what return they deserve for putting their money at risk. Our new model is no doubt helpful to better understand the relationship between risk and return under different market conditions. It can be potentially applied widely, for example, it may be useful both for market investors and financial risk managers in their investment/management decision-making

Autoregressive Diffusion Model for Graph Generation

Diffusion-based graph generative models have recently obtained promising results for graph generation. However, existing diffusion-based graph generative models are mostly one-shot generative models that apply Gaussian diffusion in the dequantized adjacency matrix space. Such a strategy can suffer from difficulty in model training, slow sampling speed, and incapability of incorporating constraints. We propose an \emph{autoregressive diffusion} model for graph generation. Unlike existing methods, we define a node-absorbing diffusion process that operates directly in the discrete graph space. For forward diffusion, we design a \emph{diffusion ordering network}, which learns a data-dependent node absorbing ordering from graph topology. For reverse generation, we design a \emph{denoising network} that uses the reverse node ordering to efficiently reconstruct the graph by predicting the node type of the new node and its edges with previously denoised nodes at a time. Based on the permutation invariance of graph, we show that the two networks can be jointly trained by optimizing a simple lower bound of data likelihood. Our experiments on six diverse generic graph datasets and two molecule datasets show that our model achieves better or comparable generation performance with previous state-of-the-art, and meanwhile enjoys fast generation speed.Comment: 18 page

End-to-End Stochastic Optimization with Energy-Based Model

Decision-focused learning (DFL) was recently proposed for stochastic optimization problems that involve unknown parameters. By integrating predictive modeling with an implicitly differentiable optimization layer, DFL has shown superior performance to the standard two-stage predict-then-optimize pipeline. However, most existing DFL methods are only applicable to convex problems or a subset of nonconvex problems that can be easily relaxed to convex ones. Further, they can be inefficient in training due to the requirement of solving and differentiating through the optimization problem in every training iteration. We propose SO-EBM, a general and efficient DFL method for stochastic optimization using energy-based models. Instead of relying on KKT conditions to induce an implicit optimization layer, SO-EBM explicitly parameterizes the original optimization problem using a differentiable optimization layer based on energy functions. To better approximate the optimization landscape, we propose a coupled training objective that uses a maximum likelihood loss to capture the optimum location and a distribution-based regularizer to capture the overall energy landscape. Finally, we propose an efficient training procedure for SO-EBM with a self-normalized importance sampler based on a Gaussian mixture proposal. We evaluate SO-EBM in three applications: power scheduling, COVID-19 resource allocation, and non-convex adversarial security game, demonstrating the effectiveness and efficiency of SO-EBM.Comment: NeurIPS 2022 Ora

MUBen: Benchmarking the Uncertainty of Pre-Trained Models for Molecular Property Prediction

Large Transformer models pre-trained on massive unlabeled molecular data have shown great success in predicting molecular properties. However, these models can be prone to overfitting during fine-tuning, resulting in over-confident predictions on test data that fall outside of the training distribution. To address this issue, uncertainty quantification (UQ) methods can be used to improve the models' calibration of predictions. Although many UQ approaches exist, not all of them lead to improved performance. While some studies have used UQ to improve molecular pre-trained models, the process of selecting suitable backbone and UQ methods for reliable molecular uncertainty estimation remains underexplored. To address this gap, we present MUBen, which evaluates different combinations of backbone and UQ models to quantify their performance for both property prediction and uncertainty estimation. By fine-tuning various backbone molecular representation models using different molecular descriptors as inputs with UQ methods from different categories, we critically assess the influence of architectural decisions and training strategies. Our study offers insights for selecting UQ and backbone models, which can facilitate research on uncertainty-critical applications in fields such as materials science and drug discovery